Attorney's Docket No.: 07844-338001 / P3 12 



APPLICATION 
FOR 

UNITED STATES LETTERS PATENT 



TITLE: GRAPHICALLY REPRESENTING DATA VALUES 

APPLICANT: ANDREI M. HERASIMCHUK 



CERTIFICATE OF MAILING BY EXPRESS MAIL 
Express Mail Label No. EM43293 1 145US 

February IS. 2001 

Date of Deposit 



PATENT 

ATTORNEY DOCKET NO: 07844/338001 



GRAPHICALLY REPRESENTING DATA VALUES 
CLAIM OF PRIORITY 
This application claims priority under 35 USC § 119(e) to U.S. Patent Application 
Serial No. 60/183,174, filed on February 17, 2000, the entire contents of which are 
hereby incorporated by reference. 

BACKGROUND 

This invention relates to the graphic representation of data values. 

Digital images used to represent and convey information to a computer user may 
take a variety of forms such as alphanumeric characters, graphs, and pictures. Digital 
images may be stored digitally, manipulated, and then rendered on video monitors or 
printers. 

Digital images may have a variety of properties such as color, texture, brightness, 
contrast, and may be associated with nonvisual properties such as speed of motion or 
sound effects. The properties of an image may be defined when the image is created or 
modified and may be stored with the image. 

Image properties are generally quantifiable. For example, the brightness of each 
pixel or other part of an image may be expressed as a quantity that may vary between 
0% white (i.e., 100% black) and 100% white. Quantifiable image properties may be 
modified through mathematical translations of the data values that define the image 
properties of the parts of the image. In some systems, the user may alter the data values 
defining one or more image properties by using a keyboard or other input device. In 
other systems, a user may modify image properties through a graphical user interface 
(GUI). For example, a range of possible data values may be displayed graphically to the 
user and the user may select the desired new data value with a mouse or other pointing 
device. 

One particular GUI used to alter image properties uses a control tab of the kind 
shown in FIG. 2. A square 200 representing an x-y coordinate system is displayed to the 
user. The range of base data values for the image property (image brightness is shown in 
FIG. 2) is displayed graphically along the bottom horizontal edge (the x-coordinate) 210 
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of the square, while the range of possible new data values is displayed graphically along 
the left vertical edge (the y-coordinate) 220 of the square. The diagonal line 230 from the 
bottom left to the top right corner of the square represents a base state in which each base 
data value is identical to the new data value (i.e., x=y), indicating that no mathematical 

5 transformation has yet been performed. Each point on the line represents all of the pixels 
(or other parts) of the image that have the brightness value indicated on the x-axis. 

To alter the image property the user may select a point on the diagonal line (e.g., 
point A) and drag it to a new position within the square (e.g., point N). Software then 
generates a new curve 240 that passes through the endpoints of the diagonal line and the 

10 new position of the point. The shape of curve 240 depends on choices made by the 
software designer. The curve 240 defines a mathematical translation of old data values 
into new data values. Every point along curve 240 has an x-y coordinate, where the 
horizontal, x-coordinate represents the old data value and the vertical, y-coordinate 
represents a corresponding new data value. Thus, when the transformation defined by the 

15 new curve is applied, every pixel having an old brightness value of x will assume a new 
brightness value of y. In this way, the brightness characteristics of the whole image can 
be altered. For example, dark pixels can be made lighter. 

The GUI shown in FIG. 2 can be confusing to a user. The diagonal line indicating 
the base state of no change has a constant slope that users may interpret as implying a 

20 constantly increasing change of pixel brightness rather than a lack of change, because 

rising lines in x-y coordinate systems usually represent constant change. Moving point A 
to destination N (as indicated by arrow 225) to create a new curve may give the 
impression that the old data value associated with point A (the x-coordinate of point A) 
has acquired the new data value associated with point N (the y-coordinate of point N). In 

25 fact, it is the old data value associated with point B, located directly below point N, that 
has acquired the new data value associated with point N, because the x-coordinates of 
points N and B are identical. 

SUMMARY OF THE INVENTION 
30 In general, in a first aspect, the invention features a method of graphically 

representing base data values and possible data values associated with a quantifiable 
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image property including the steps of displaying to a user a line that represents a range of 
base data values, displaying to a user a two-dimensional coordinate space in which the 
two dimensions are respectively parallel to and normal to the line, and representing a 
range of possible data values by distances along the normal dimension of the coordinate 

5 space from corresponding base data values on the line. 

Implementations of the invention may include the following features. The line 
may be displayed horizontally and the coordinate space may be skewed from the 
horizontal direction. The two-dimensional coordinate space may be a polygon, wherein 
two nonadjacent vertices of the polygon coincide with the ends of the horizontal line in 

10 the two-dimensional coordinate space. The polygon may be a parallelogram, wherein the 
parallelogram includes a first vertical side parallel to a second vertical side and a first 
skewed side parallel to a second skewed side. The possible data values represented by 
endpoints of the first vertical side and the second vertical side may include the minimum 
and maximum of the range of possible data values, wherein the possible data values 

15 represented by points on the first skewed side comprise the minimum of the range of 
possible data values, and wherein the possible data values represented by points on the 
second skewed side comprise the maximum of the range of possible data values. The 
data values may be used in a computer graphics display, and the quantifiable image 
property may include brightness or color. 

In another aspect, the invention features a method of interactively transforming a 
graphical representation of data values associated with a quantifiable image property 
including the steps of displaying to a user a line that represents a range of base data 
values, displaying to a user a two-dimensional coordinate space in which the two 
dimensions are respectively parallel to and normal to the line, representing a possible 
range of data values by distances along the normal dimension of the coordinate space 
from corresponding base data values on the line, and interactively providing a curve in 
the coordinate space, a change in each base data value corresponding to the distance 
along the normal dimension of the coordinate space from the curve to the point on the 
line representing the base data value. 

20 Implementations of the invention may include the following features. 

Interactively providing a curve in the coordinate space may include selecting a point on 
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the line, dragging the point to a new position in the coordinate space, and defining a 
curve in the coordinate space through the ends of the line and through the new position of 
the point. The tangent to the curve at the new position of the point may be parallel to the 
line. Interactively transforming a graphical representation of data values associated with 

5 a quantifiable image property may further include displaying to a user the base data value 
of the selected point, and displaying to a user the new data value of the point as it is 
dragged. One or more nodes in the coordinate space may be defined and a curve in the 
coordinate space through the ends of the line and through the nodes may be defined. The 
tangent to the curve at each node may be parallel to the line. 

10 In a further aspect, the invention features method of graphically representing old 

data values and new data values, the data values being associated with a quantifiable 
image property, including displaying to a user a line that represents a range of old data 
values, displaying to a user a two-dimensional coordinate space in which the two 
dimensions are respectively parallel to and normal to the line, representing new data 

15 values with a curve in the coordinate system, and displaying a line along the normal 
dimension of the coordinate system between an externum of the curve and the line. 

DESCRIPTION OF THE DRAWINGS 
FIG. 1 is a block diagram of a computer system suitable for implementing the 
20 invention. 

FIG. 2 is a prior art graphical user interface used for modifying image properties. 
FIG. 3 is a graphical user interface used for modifying image properties in 
accordance with the invention. 

FIG. 4 is a flowchart of a method in accordance with the invention. 

25 

DETAILED DESCRIPTION 
As shown in FIG. 3, a GUI display image 300 includes a square 302 that defines 
an x-y coordinate system used for modifying the data values associated with quantifiable 
image properties. The x-coordinate runs along the horizontal direction of square 302 and 
30 the y-coordinate runs along the vertical direction of the square, although (unlike a 

conventional rectilinear coordinate system) a given value of y occupies a vertical position 
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that depends on its location along x-axis. The square 302 contains an inactive 
background region 310 and an active region 3 12 in the shape of a parallelogram that is, 
for example, lighter in color and therefore visually distinguishable from inactive 
background region 310. Two sides 303 of the parallelogram active region 312 are 

5 parallel to the vertical sides 304 of square 302, while two sides 305 of parallelogram 
active region 312 are skewed from the direction of the horizontal sides 306 of the square. 
A straight, horizontal line 315 in active region 312 runs parallel to the horizontal 
sides 306 of square 302, between the two vertices 317 that form the obtuse angles of 
parallelogram active region 312. (In this context, the term "line" refers a line segment, a 

10 straight segment of finite length.) 

The straight, horizontal line 315 represents a range of base data values assigned to 
the image property to be modified (for example, the brightnesses of pixels in an image), 
and active region 312 represents the parameter space of data values that may be assigned 
when the base range of data values is modified. The x-coordinate 331 of a point on 

15 line 315 corresponds to a base data value within the range of base data values assigned to 
an image property to be modified. The y-coordinate in active region 312 directly 
above 333 or below 335 a point on line 315 corresponds to a possible new data value that 
may be assigned to the point when the image property is modified. 

Horizontal line 315, representing base data values for an image property, 

20 combined with parallelogram active region 312, representing possible new data values for 
an image property, provides an intuitive graphical space for translating base data values 
into new data values. The ends of horizontal line 315 represent the extrema of a range of 
base data values. For example, in FIG. 3, where brightness is the image property to be 
modified, the ends of horizontal line 315 correspond to 0% and 100% brightness, and 

25 points on line 315 between the ends correspond to intermediate brightness values. Since 
the points at the ends of horizontal line 315 correspond to the extrema of the range of 
base data values, upon modification the point at one end of the line 315 may only acquire 
a larger value and the point at the other end of the line 315 may only acquire a smaller 
value. Thus, at one end of horizontal line 315 available space in active region 312 exists 

30 only above point 318, because the data value assigned to this point can only increase, 
while at the other end of line 315 space in active region 312 exists only below point 319, 
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because the data value assigned to this point can only decrease. Graphically, this requires 
that parallelogram vertices 317 coincide with the ends of horizontal line 315. 

Points on horizontal line 315 between the ends 3 17 of the line correspond to base 
data values intermediate between the minimum and maximum of the range of values. 

5 Since they are intermediate values, they may be either increased or decreased when 
modified and, therefore, space exists in active region 312 both above and below line 315 
for every point on the line except for the endpoints 317. For example, in FIG. 3, the 
point 318 at the left end of line 315 represents a data value indicating minimum 
brightness, so active space only exists above point 318, while the point 319 of at the right 

1 o end of line 315 represents a data value that indicating maximum brightness, so active 
space only exists below point 319. The point in the middle of line 315 represents 50% 
brightness and can increase to 100% brightness or decrease to 0% brightness, which is 
why the active space above it is equal to the active space below it. 

A graphical bar 320 representing the range of data values currently assigned to the 

15 image property runs parallel to the bottom edge 306 of the square 302. For example, in 
FIG. 3, the image property to be changed is brightness, and graphical bar 320 displays the 
range of brightness from 0% brightness at the left end of graphical bar 320 to 
100% brightness at the right end of graphical bar 320. 

In addition to horizontal bar 320 in image 300, vertical graphical bars 330, 

20 representing the range of possible data values that may be assigned to the image property 
when it is modified, run parallel to the vertical sides 304 of square 302. Vertical 
graphical bars 330 do not run the entire length of the vertical sides 304 of the square. 
Rather, they run parallel to the vertical sides of parallelogram active region 312. 
Graphical bar 331 becomes brighter as it goes up because the brightness of point 318 can 

25 only increase, while graphical bar 332 becomes darker as it goes down because the 
brightness of point 319 can only decrease. 

A nonorthogonal grid may be drawn inside active region 312. Vertical grid 
lines 352 parallel to the sides 304 of square 302 indicate lines of constant base data 
values. Skewed lines 354, parallel to the skewed sides of parallelogram active region 312 

30 indicate lines of constant possible data values that the base data values may acquire 
through a modification of an image property. For example, in FIG. 3, line 353 at the 
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bottom of parallelogram active region 312 indicates 0% brightness, line 355 at the top of 
parallelogram active region 312 indicates 100% brightness and line 354 midway between, 
and parallel to, lines 353 and 355 indicates 50% brightness. 

A user may modify the data values associated with a quantifiable image property 

5 by altering the horizontal line 315. A user may select a starting point, e.g., point A, on 
horizontal line 315 with a mouse or other pointing device and drag the point to a new 
location in active region 3 12, e.g., point N. When the user stops dragging the starting 
point and releases it in active region 312, a new line 360 (which in general may be a 
curved segment of essentially any shape) is created and drawn though the ends 317 of 

10 line 315 and the new position, point N, of the point the user has dragged. In one 

implementation, the tangent to new line 360 at point N is parallel to horizontal line 315. 
New line 360 defines the modification of the data values associated with the quantifiable 
image property, in that the x-coordinate of a point on new line 360 corresponds to the old 
data value and the y-coordinate of the point on new line 360 corresponds to the new data 

1 5 value. Thus, for example, dragging starting point A on line 3 1 5 in FIG. 3 to point N on 
new line 360 causes grays of 25% brightness in the original image to acquire a value of 
approximately 66% brightness. 

Further referring to FIG. 3, when a user drags a point on line 3 1 5 to a new 
location, e.g., point N, a vertical line 370 perpendicular to line 315, from point N to 

20 line 315 can be displayed to the user. Vertical line 370 serves to remind the user that it is 
the x-coordinate of point N, rather than the x-coordinate of starting point A, that 
corresponds to the old data value when the image property is modified. A dot 372 at the 
base of vertical line 370 on line 315 can be displayed to further reinforce this notion. 
New line 360, passing through end points 317 of line 315 and point N, may be 

25 displayed to the user while the starting point is being dragged and before the starting 

point is released by the user. New line 360 will necessarily change as the starting point is 
dragged to different locations within active region 312 until the point is released. 
Additionally, the value of the old data value and the new data value it will acquire when 
the starting point is released may be displayed either through graphical aids 380 or 

30 through a numerical display 382. For example, in FIG. 3, both graphical aids 380 and 
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numerical display 382 indicate that moving starting point A to point N results in changing 
grays of 25% brightness to grays of 66% brightness. 

As shown in FIG. 4, data values corresponding to an image property may be 
modified with the use of an x-y coordinate system that includes a skewed active region 

5 and a horizontal line through the active region 400. The coordinate system, the active 
region and the horizontal line are displayed to a user 400. The user selects a point on the 
horizontal line 410, through the use of a mouse or other pointing device, which may 
include keyboard commands. The user then drags the selected point to a new location 
within the active region 420. While the selected point is dragged, a vertical line may be 

10 drawn from the point to the horizontal line. The user releases the selected point 430 at 
the desired destination point and a new line, passing through the ends of the horizontal 
line and through the destination point, is defined 440. Data values corresponding to the 
image property are redefined according to the shape of the new line 450. 
Other embodiments are within the scope of the following claims. 

15 For example, when an image property is modified by modifying the horizontal 

line and creating a new line, the new line need not be defined by only the endpoints of the 
horizontal line and a single point dragged by the user. More than one point on the 
horizontal line may be dragged to new positions and the new line may be defined to pass 
through all of the new positions of the points in addition to the endpoints of the horizontal 

20 line. The new line may also be defined by use of control points that the new line does not 
pass through. Furthermore, positions of points through which new line 360 passes need 
not be defined by dragging points from line 315, but may be defined by other selection 
methods, such as with a mouse or other such pointing device or by entering the new 
point's coordinates with, e.g., a keypad. The multiple new positions of points selected by 

25 the user may define the nodes of the new line that passes through them and define the 
shape of the new line. 

The horizontal line that represents the range of base values need not be strictly 
horizontal. It can, for example, be vertical or at some angle to vertical or horizontal that 
is close enough to vertical or horizontal to provide the same benefits. In other 

30 arrangements, the line can be at any angle to horizontal or vertical if the skewed region 
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